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ABSTRACT 

In its third year, Project CELSIM (career Education 
and Learning Strategies Implementation Model) of the New York City 
Public Schools served 282 Hispanic high-school students with limited 
English proficiency. The project provided students with 
English-as-a-Secjnd-Language and Native Language Arts instruction, 
bilingual instruction in content area subjects, career awareness 
instruction, and counseling services. The project also provided staff 
with a series of training conferences on teaching methodologies and 
learning styles. Strengths of the project included xts information 
sharing with mainstream staff and its thrust toward placing students 
in mainstream bilingual activities. This report examines the 
project* s staffing, implementation, outcomes by individual 
objectives, and a case history. Appendices describe data collection 
and analysis procedures and list instructional materials used in the 
project. ( JDD) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 
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FOREWORD 



Tbe body of this report is preceded by an Extract which presents an overview of saltern points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about tbe 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Prop-am Assessment . This includes such 
information as staffing, program implementation, and outcome and implementation objectives. 
Instructional objectives are presented first, followed by noninstructional objectives. The report then 
addresses those aspects of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in A ppendix A following the text. 
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EXTRACT 

PROJECT DIRECTOR: Helen Corchado FUNDING CYCLE: Year 3 of 3 

SITE 



School Borough Program Service Category 



Theodore Roosevelt High School Bronx BIS I, II 

James Monroe High School Bronx BIS I, II 

William H. Taft High School Bronx BIS I, II 

♦The project enrolled 282 students (145 more than in the previous year), serving 208 in the fall and 208 in the 
spring. 

STUDENT BACKGROUND 

Native Language Number of Students Countries of Origin Number of Students 

Spanish 282 Puerto Rico 

Unites Slates 
Dominican Republic 
Mexico 
Guatemala 
Honduras 
Other countries 

Mean Years of Education in Native Country : 3.9; in the United States: 5.8 
Percentage of Students Eligible for Free Lunch Program : 81.9 



ADMISSION CRITERIA 

The project admitted bilingual special education students in BIS I and II who had not previously received 
services funded by Title VII and who had scored at or below the 40th percentile on the Language Assessment 
Battery (LAB), classifying them as limited English proficient (LEP). Project students also had gaps in their 
schooling. 



PROGRAMMING 
Features 



Project CELSIM provided LEP special education students with English as a Second Language (E.S.L.), 
Native Language Arts (N.L.A.), bilingual instruction in the content area subjects, career awareness instruction, 
and counseling services. The project also provided staff with a series of training conferences on teaching 
methodologies and learning styles. 



Enrollment* 
(fall) (spring) 
96 100 
52 49 
60 59 



154 
50 
45 
3 

2 
3 

25 



Strengths and Limitations 

Information-sharing with mainstream staff was a plus for the project, as was its thrust toward placing 
students in mainstream bilingual activities. 

A limitation was the lack of a teacher trainer-the project director doubled in this capacity. 

CONCLUSIONS AND RECOMMENDATION 

Project CELSIM was fully implemented. LEP Hispanic special education students at each of the three 
project sites received E.S.L., N.L.A., bilingual content areas, and career awareness instruction as well as 
counseling services. Program staff participated in a series of workshops and conferences on teaching techniques 
and learning strategies. 

The project met its objectives for N.L.A and guidance. For lack of time and staff, its objective for 
curriculum development was only partially met, although 10 of 12 projected instructional units were completed. It 
failed to meet its objectives for E.S.L., career awareness, and dropout rate. While it also failed to meet the 
content area objective, students show* d improvement from fall to spring in all areas and did meet the objective in 
business. ORF.A was unable to evaluate the objective for staff development because the project did not provide 
the necessary data. OREA could not evaluate the objective for self-esteem because of the low number of scores 
submitted. 

The conclusion, based on the findings of this evaluation, lead to the following recommendations: 

• If funds permit, recruit a teacher trainer. 

• The program should provide OREA with data necessary to evaluate all objectives. 

• Explore reasons why the percentage of students meeting the E.S.L. objective was less than half of 
what was expected. 

• Explore more ways in which to achieve the content area objectives. 

• Explore reasons wb ; project students continue to drop out of school at the same or higher rate than 
mainstream students. 



PROGRAM ASSESSMENT 



STAFFING 

Title VII Staff (Total 2) 



Degree Language Competencies 



Project Director* M.A. Spanish 

Curriculum & Career Specialist M.S. Spanish 

*The project director also acted as the teacher trainer. 
Other Staff Working With Project Students (Total 20) 



Degree 




Certification* 


B.A. 


2 


Special Education 


B.S. 


2 


Special Education Common Branches 


M.A. 


11 


Bilingual Special Education 


M.S. 


5 


High School English and E.S.L. 






Bilingual Special Education and 



9 
7 
2 
1 



Funding 

Full time 
Part time 



Teaching/Communicative 
Proficiencies (TP/CP) * 

Spanish TP 13, CP 3 



Certifications are not exclusive. 

IMPLEMENTATION AND OUTCOMES (Objectives prefaced by •) 
English as a Second Languaee (E.S.L.) 

E.S.L. was offered on the beginning, intermediate, and advanced levels five times a week at both sites. 
All levels used several instructional strategics, iacluding the communicative and whole-language approach, 
content-based E.S.L., and the assistance of a bilingual paraprofessional. Teachers were encouraged to focus on 
student-centered activities. 



•Teaching Proficiency' (TV): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 



• By the last year of the program, 65 percent of the students will make meaningful gains in English 
language achievement. 

Evaluation Instrument: Language Assessment Battery (LAB)' 

Pretest: March 1990; posttest: March 1991 

Number of students for whom pre- and posttest data were reported: 112 
Percentage of students showing gain: 27.7 
Project did not meet objective. 

Native Language Arts 

Students lacking literacy skills in their native language: 20 percent (estimated). 

All native language instruction used the whole language approach, the language experience chart 
(students were asked to make up a story), phonetics, and the read-response method (in which students related 
stories to their personal life). 

Students were offered five periods of N.L.A. a week on the beginning and intermediate level at William 
Taft and James Monroe High Schools and on the beginning, intermediate, and advanced levels at Theodore 
Roosevelt High School. 

# As a result of participating in the program, students will show meaningful gains in Spanish reading 
achievement. 

Evaluation Instrument: La Prueba de Lectura 

Number of students for whom pre- and posttest data were reported: 28 

Mean gain = 1.4 N.C.E.s (s.d.= 3.0) and was statistically significant (t=2.43, p_<.05). 

Project met objective. 



Content Area Subjects 

Project students took courses in mathematics, science, social studies, and business. All courses were 
taught in Spanish except for fundamentals of mathematics, which was taught in English supplemented by Spanish. 
It was difficult for students to understand abstract concepts in mathematics. The project director reported that 
she had trouble finding other content area materials at the reading level of the students and relied on teacher- 
made materials. 



•OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students' 
performance on the 17VB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, anu OREA used instead the group on which the LAB was normed. 
Test scores are reported in Normal Curve Equivalents (N.C.E.s), which are normalized standard scores with a mean 
of 50 and a standard deviation of 21.06. It is assumed that the norm group has a zero gain in N.C.E.s in the absence 
of supplementary instruction and that participating students' gains are attributable to project services. 
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For a list of instructional materials used, see Appendix B. 

• At least 70 percent of the students enrolled in content area classes (science, mathematics, social 
studies, and business) will score at or above the passing criterion of 65. 

Evaluation Indicator: Final course grades. 

Passing Grades in Content Area Courses 

Fall Spring 

Number of Students PercentaEe Number of Students Percentage 

Mathematics 177 

Science 127 

Social studies 140 

Business 122 

Project did not meet objective except for business in the spring* (However, there was noticeable 
improvement from the fall to the spring semesters.) 

Career Awareness 

• As a result of participating in the career awareness/exploration class, 80 percent of the participating 
students will show significant improvement in occupational aspiration. 

Evaluation Instrument: Maier Occupational Aspiration Scale. 

The proposed instrument docs not exist, and the project developed its own test to measure career 
awareness and occupational aspirations and administered it on a pre-/positesi basis. 

Number of students for whom pre- and posttcst data were reported: 27 

Mean gain = 4.8 scale points (s,d.= 18.9) and was not statistically significant (1=1.32)- 

Project did not meet objective. 

Guidance 

• The bilingual counselor will have developed a cognitive map for each participating student. 
Evaluation Indicator: Individual Educational Plan (I.E.P.). 

The School-Based Support Team (S.B.S.T.) developed an l.E.P, (or cognitive map) for each of the project 
students. The project's part-time bilingual counselors did not participate in this, since they were unable to 
provide the necessary lime. They did, however, participate in student evaluation. 

Project met objective. 

Self-Esteem 

• As a result of participating in the program, 80 percent of the target students will show a statistically 
significant growth in self-esteem. 

Evaluation Instrument: Self-Perception Inventory Form. 



58.2 141 63.8 

59.1 85 64.7 

50.7 110 58,2 

59.8 78 105 



The program distributed lie Self-Perception Inventory Form in Spanish and English on a pre- and post- 
test basis. 

Number of students for whom gain or loss were reported: 21 
Mean gain ■ 14.52 points (s.d.<= 10.1) 
Greatest reported loss = 4 points 
Greatest reported gain = 28 points 

OREA could not evaluate this objective as stated because of the low number of scores submitted. 
Dropout Rate 

• Program students will have a significantly lower dropout rate than similar non-program students. 
Evaluation Indicator: School records. 

Dropout Rate 

Program Students Mainstream Students 

Number of Students Percentage Dropout Rate Difference 

Theodore Roosevelt High School 18 14.4 7.6 6.8 

James Monroe High School 7 9.1 9.8 -0.7 

William H. Taft High School 5 63 8.2 -1.9 

The dropout rate of project students was not significantly bwer that of mainstream students and at one 
site, Theodore Roosevelt High School, the dropout rate was much higher. 

Project did not meet objective. 

Staff Development 

• Ninety percent of the project staff members attending teacher training conferences and workshops 
will show a statistical!) signify., it increase in knowledge relating to career infusion techniques and 
learning strategies. 

Evaluation Indicator: Project-developed teacher skills questionnaire. 

Project CELSIM presented demonstration classes in E.S.L. that were videotaped for discussion and 
review. The project also held seven teacher training conferences, covering the following: "Using Video for^ 
Language Development," "E.S.L. Methodology in the Math Class," "Learning Styles in the E.S.L classroom," 
"Communicative Activities in the Classroom," "E.S.L. Methodologies in the Social Studies Class," and 
"Communication Activities in the E.S.L. Classroom." 

An evaluation questionnaire was distributed among the participating teachers at the end of each 
conference. While 84.8 percent of the respondents to the evaluation questionnaire at the second conference 
indicated that they thought the workshop was helpful, and 90.9 percent believed that they would be able to apply 
what they had learned, the project did not report on the development of skills. 

OREA could not evaluate objective. 
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Curriculum pevelopment 



• The curriculum specialist will have developed 12 instructional units in each of the content areas, 
utilizing learning strategies and career infusion technologies- 
Evaluation Indicator: Project-developed instructional material inventory. 

The curriculum specialist developed six instructional units during the first and second years and four more 
(one in science, two in health, and one in social studies) b the year under review. The project director reported 
that lack of time and staff prevented the last two instructional units from being completed. 

Project partially met objective. 



Other Activities 

Program students participated in field trips throughout the year. They visited museums, wtfct to the 
theater, visited the Statue of Liberty, and visited 1NFOQUEST at the l.B.M. building. 



MAINSTREAM 1NG 

Eleven students (3.9 percent of participants) were mainstreamed. 



Academic Achievement of Fonrer Students in Mainstream 

Thirteen of the previoui year's participants were mainstreamed. However, the project reported grades for 
only a few. Their subsequent performance is tabulated below: 

Mainstream Students Enrolled Students Ach ieving Passing Grades 

English 
Mathematics 
Science 
Social Studies 
Business/Vocational 



4 

3 
4 
4 

3 



1 
2 
2 
4 
3 



CASE HISTORY 

M. was a 16-year old Hispanic female who was in a BIS 1 program when the project started. Her first 
language was Spanish, but she seemed to have a good command of social English. Her interest in class work was 
erratic, and school was mostly a social activity for her. She was not mandated for counseling by the Committee 
for Special Education (C.S.E.) but visited the counselor frequently to discuss her boyfriends, marriage plans, etc. 
These stories became more exaggerated and bizarre, and the counselor contacted the parent. It was learned that 
M had been sexually abused by an uncle and had serious problems at home. The counselor made a referral to 
the S B S T for academic and psychological evaluation, contacted Lincoln Hospital for psychiatric assistance, and 
arranged a conference with a doctor, the student, and the mother. To increase M.'s interest in school she was 
recommended for the COOP program (in which students attend school and work on alternate weeks). Teachers 
were advised that M.'s stories were fantasy and that she was receiving guidance. In her E.S.L. classes, the project 
suggested to the teacher that group activities be used to take advantage of M.'s sociability, allowing her to interact 
with the other students. 
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APPENDIX B 



Ins, oictional Materials 

Social Studies 

Comprende tu Mundo, Jarret Publishing. 
Europa. Asia. Africa, Australia . Silver Burden. 
Vavamos pot el Mundo . Lectorum. 



Science 

» Horizontes en Cienria . Silver Burden. 

Las Cieocias BiokSgicas . Publisha Disiribuidora Escolar, Puerto Rico. 
The Wonders of Science, Steck Vaughn. 



Mathematics 

Malematicas - Exploremos tu Mundo , Silver Burdett. 
Language Development Mathematics, Book A , Addison Wesley. 

Native Language Arts (N.L.A.) 

Historietas en Espafiol . National Textbook G Tp»ny. 

Cuentos Simpatioos, National Textbook Company. 

Cuentos Corrientes , National Textbook Company. 

Escribir para Leer . Santillana. 

Spanish is Fun , Amsco. 

Colecckm de Emflio Salgieri , Lectorum. 

Aventuras de Don Quiiote , Regents. 
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APPENDIX A 



DATA COLLECTION AND ANALYSIS 

Student Data Form 

This one-page form is fiiled out by staff for each particqming and mainstreamed student. OREA gathers 
data from this form on backgrounds, tomographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
response s to the Questionnaire. 

Citywide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.C.E-s). N.C.E.s are 
normalized standard scores with a mean of 50 and a standard deviruion (s.d.) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.C.Es is the same 
whether it is at the lower or the higher end of the scale. N.C.E.s can be used in arithmetic computations to 
indicate group progress. (Percentile scales, although more familiar to many, are unsuitable for such computations 
since they are not equal-interval) 

Likert-Type Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student atfitudes (i.e., toward school and career, native language use, and native 
and mainstream cultures). They also assess staff and parent altitude and reactions to workshops and other 
activities. 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where all students of limited English proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students umld be attributed to project services. 
(See "Citywide Test Scores" above.) 

To lest whether pre/posltesi gains are greater than could be expected by chance alone, OREA uses a 
West. To determine whether a difference between two proportions (e.g., program and mainstream attendance 
rates) is greater than could be expected by chance, OREA uses a z-lesl. The level of significance is set at .05 for 
all tests. 




